Bayesian Sensor Image Fusion using Local Linear Generative Models

We present a probabilistic method for fusion of images produced by multiple sensors The approach is based on an image formation model in which the sensor images are noisy locally linear functions of an underlying true scene latent variable A Bayesian framework then provides for maximum likelihood or maximum a posteriori estimates of the true scene from the sensor images Least squares estimates of the parameters of the image formation model involve local second order image statistics and are related to local principal component analysis We demonstrate the e cacy of the method on images from visible band and infrared sensors Introduction Advances in sensing devices have fueled the deployment of multiple sensors in several computational vision systems for example Using multiple sensors can increase reliability with respect to single sensor systems This work was motivated by a need for an aircraft autonomous landing guidance ALG system that Author is now with Digimarc Corporation SW nd Ave Suite Tualatin Oregon uses visible band infrared IR and radar based imaging sensors to provide guidance to pilots for landing aircraft in low visibility IR is suitable for night operation whereas radar can penetrate fog The application requires fusion algorithms to combine the di erent sensor images Images from di erent sensors have di erent characteristics arising from the varied physical imaging pro cesses Local contrast may be polarity reversed between visible band and IR images A particular sensor image may contain local features not found in another sensor image i e sensors may report comple mentary features Finally individual sensors are subject to noise Fig a and b are visible band and IR images respectively of a runway scene showing polarity reversed rectangle and complementary circle features These e ects change from region to region according to the local mapping from the scene to the sensor images and pose di culties for fusion